Head mounted display apparatus

ABSTRACT

Provided regarding a technique of an HMD is a technique capable of realizing high functionality and weight reduction in a well-balanced manner and improving usability for a user. An HMD of an embodiment is a head mounted display apparatus, includes: a head attached part that is attached to a user&#39;s head and has a display screen for displaying an image; and a body trunk attached part communicating with the head attached part and attached to a portion of the user&#39;s body trunk, measures a relatively positional relationship between the head attached part and the body trunk attached part and grasps states including a position and a direction of the head attached part states including a position and a direction of the body trunk attached part based on the positional relationship.

TECHNICAL FIELD

The present invention relates to a technique for a head mounted displayapparatus (HMD).

BACKGROUND ART

An HMD worn on a user's head can display an image of a virtual object orthe like on a transparent or non-transparent type display surface. Aconventional HMD has, as a method of displaying an image on a displaysurface, a method of displaying the image at a position aligned with aworld coordinate system or a method of displaying the image at aposition aligned with a direction of a user's head (corresponding HMD).Further, as another method, a method of displaying the image at aposition aligned with a direction of a body trunk such as a user's bodyhas also been proposed.

As an example of a conventional technique related to the HMD, JapanesePatent Application Laid-Open No. 2019-28638 (Patent Document 1) isgiven. Patent Document 1 discloses: a head mounted display or the likethat includes a head sensor for detecting a direction of a head, a bodytrunk sensor for detecting a direction of a body trunk, and the like;and a construction of a virtual desktop that is composed of two layersof a body trunk layer and a head layer and configures a continuousinformation display region in a virtual space around a wearer.

RELATED ART DOCUMENTS Patent Documents

-   Patent Document 1: Japanese Patent Application Laid-open No.    2019-28638

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

Weight of the HMD tends to increase with sophistication of its function.The weight of the HMD brings an increase of physical burden on the user,thereby making it difficult to use the HMD for a long period of time,for example. In order to reduce the physical burden on the user, it isrequired to reduce the weight of the HMD as much as possible.

As an example of the sophistication of the HMD's function, given is afunction of controlling display of an image onto a display surfaceaccording to a user's state. For example, given is a function that canuse both image display aligned with a direction of the user's head andimage display aligned with a direction of a body trunk. In an example ofPatent Document 1, the direction of the body trunk, which is measuredfrom a sensor attached to the body trunk of the user, is used as animage display reference. In the example of Patent Document 1, sensorssuch as 3-axis angular velocity sensors are provided in both the HMD ofthe head and the body trunk of the user so as to be able to detect twodirections of the head direction and the body trunk direction.

In order to sophisticate the functions of the HMD, various devicesincluding sensors need to be mounted. However, as the heavier devicesor/and a larger number of devices are mounted, the weight of the HMDbecomes heavier. In the example of Patent Document 1, the same type ofsensors needs to be provided doubly, so that there is room forimprovement in such a configuration from the viewpoint of effective useof the sensors. In mounting the HMD, designing in a well-balanced manneris required from the viewpoint of advanced functions and weight.

An object of the present invention is to provide, regarding a techniquefor a HMD, a technique that can realize advanced functions and weightreduction in a well-balanced manner and improve usability of a user. Aproblem etc. other than the above will be shown in embodiments forcarrying out the invention.

Means for Solving the Problems

A typical embodiment of the present invention has the followingconfigurations. A head mounted display apparatus of one embodiment is ahead mounted display apparatus capable of displaying an image in auser's field of view, and it includes: a head attached part that isattached to a user's head and has a display surface for displaying theimage; and a body trunk attached part that communicates with the headattached part and is attached to a part of the user's body trunk,calculates a relatively positional relationship between the headattached part and the body trunk attached part, and grasps, based on thepositional relationship, states including a position and a direction ofthe head attached part and states including a position and a directionof the body trunk attached part.

Effects of the Invention

According to a typical embodiment of the present invention, the advancedfunctions and the weight reduction can be realized in a well-balancedmanner, and the user's usability can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing a configuration of a display system including ahead mounted display apparatus (HMD) according to a first embodiment ofthe present invention;

FIG. 2 is a view showing a configuration of the HMD according to thefirst embodiment;

FIG. 3 is a view showing a configuration of a controller and the like inthe HMD according to the first embodiment;

FIG. 4 is a view showing a main processing flow in the HMD according tothe first embodiment;

FIG. 5 is a view showing a coordinate system, distance measurement of apositional relationship, and the like in the HMD according to the firstembodiment;

FIG. 6 is a view showing calculation and the like of the positionalrelationship in the HMD according to the first embodiment;

FIG. 7 is a view showing a mounting example of elements in a neckattached part in the HMD according to the first embodiment;

FIG. 8 is a view showing a configuration example about a connecting linein the HMD according to the first embodiment;

FIG. 9 is a view showing a configuration example about a width of ahousing in the HMD according to the first embodiment;

FIG. 10 is a view showing a configuration example of separatingcomponents in the HMD according to the first embodiment;

FIG. 11 is a view showing configuration examples of display informationand coordinate system information in the HMD according to the firstembodiment;

FIG. 12 is a view showing a configuration of an HMD according to asecond embodiment of the present invention;

FIG. 13 is a view showing a configuration of an HMD according to a thirdembodiment of the present invention;

FIG. 14 is a view showing a configuration of an HMD according to afourth embodiment of the present invention;

FIG. 15 is a view showing a first state and the like of a first displaycontrol example in an HMD according to a fifth embodiment of the presentinvention;

FIG. 16 is a view showing a second state of the first display controlexample in the HMD according to the fifth embodiment;

FIG. 17 is a view showing a third state of the first display controlexample in the HMD according to the fifth embodiment;

FIG. 18 is a view showing a fourth state of the first display controlexample in the HMD according to the fifth embodiment;

FIG. 19 is a view showing, as a second display control example, adisplay example of an application icon with reference to a headcoordinate system in the HMD according to the fifth embodiment;

FIG. 20 is a view showing, as a second display control example, adisplay example of an application icon with reference to a neckcoordinate system in the HMD according to the fifth embodiment;

FIG. 21 is a view showing a third display control example in the HMDaccording to the fifth embodiment;

FIG. 22 is a view showing, as a fourth display control example, asetting example in a display coordinate system of an application in theHMD according to the fifth embodiment;

FIG. 23 is a view showing, as a fourth display control example, asetting example in a display coordinate system of an application icon inthe HMD according to the fifth embodiment;

FIG. 24 is a view showing, as a fourth display control example, asetting example in a display coordinate system of a work tool in the HMDaccording to the fifth embodiment; and

FIG. 25 is a view showing, as a fifth display control example, a conceptof an inertial coordinate system in the HMD according to the fifthembodiment.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described indetail based on the drawings.

First Embodiment

A head mounted display apparatus (HMD) according to a first embodimentof the present invention will be described with reference to FIGS. 1 to11.

An HMD according to a first embodiment has a separate typeconfiguration, and is roughly configured by two parts of a head attachedpart and a neck attached part. Components of an HMD function are mountedseparately in these two parts. For example, a display device is mountedon the head attached part, and a main sensor group, a controller, abattery, and the like are mounted on the neck attached part. Such aseparate type configuration makes it possible to reduce the number ofdevices mounted on the head attached part and omit mounting at leastsome of the sensor group required for advanced functions onto the headattached part. Consequently, this HMD can reduce weight of the headattached part attached to a user's head as compared with a conventionalHMD. Therefore, this HMD makes a wearing feeling of a user better andalso makes use thereof easy for a long time.

Further, in this HMD, a position and a direction (corresponding posture)of the neck attached part, and a position and a direction (correspondingposture) of the head attached part can be independently changedaccording to movement of the user. Therefore, this HMD has a means formeasuring a relatively positional relationship between the neck attachedpart and the head attached part. As an example of such a means, this HMDhas a distance measuring sensor on the neck attached part, and measuresa position and a direction of the head attached part from the neckattached part. Then, this HMD uses a measured positional relationship tograsp both states such as the position and the direction of the neckattached part and states such as the position and the direction of thehead attached part based on calculation of rotation and the like of acoordinate system.

This HMD uses the above positional relationship to be able to correctand convert a state (corresponding sensor data etc.) detected by asensor of one of the head attached part or the neck attached part to andinto a state (corresponding sensor data etc.) in a coordinate system ofthe other of the head attached part or the neck attached part. That is,this HMD can convert, by using the positional relationship, a state in ahead coordinate system, which is detected by a sensor of the headattached part, into a state in a neck coordinate system of the neckattached part. Further, this HMD can convert, by using the positionalrelationship, a state in the neck coordinate system, which is detectedby a sensor of the neck attached part, into a state in the headcoordinate system of the head attached part.

By such a configuration, the HMD according to the first embodiment candetect both the head direction (corresponding head-attached-partdirection) and the body trunk direction (correspondingneck-attached-part direction) of the user, and can display and control avirtual object obtained by using each of the two directions. Forexample, in this HMD, used in combination can be displaying an image ata position aligned with the head direction in the head coordinate systemand displaying an image at a position aligned with the body trunkdirection in the neck coordinate system. Such advanced functions make itpossible to improve convenience such as work support of the user.

In the HMD according to the first embodiment, a position/attitude sensoris particularly mounted on not the head attached part but the neckattached part. This position/attitude sensor is a sensor that detects aposition and a direction (corresponding posture) with reference to thebody trunk including the neck and shoulders, in other words, a sensorthat detects the state in the neck coordinate system. The state detectedby this sensor is different from a state with reference to the head(corresponding head attached part and head coordinate system) as it is.Therefore, this HMD obtains a state with reference to the headcoordinate system by calculating the coordinate system from the statedetected by the neck attached part based on the above positionalrelationship.

[Display System]

FIG. 1 shows a configuration of a display system including an HMD 1according to the first embodiment. In the HMD 1, a head attached part 1Aand a neck attached part 1B are connected by connecting lines 4. Thehead attached part 1A and the neck attached part 1B are connected so asto be capable of communication and power feeding by the connecting lines4 which are cables. Although not shown in FIG. 1, there is a user's bodyattaching the HMD 1. The head attached part 1A is attached to the user'shead, and the neck attached part 1B is attached near a user's neck andshoulders. As an option, the HMD 1 may accompany an operating tool 2which is a remote controller. The user can operate the HMD 1 with theoperating tool 2 held in his hand. For example, the neck attached part1B and the operating tool 2 perform short-range wireless communication.The HMD 1 may communicate with an external device. For example, the HMD1 may be connected to a server 3A of a business operator, a PC 3B athome, or the like via a communication network. Incidentally, separatelyfrom the coordinate system, X, Y, and Z directions may be used asexplanatory directions. The X direction corresponds to a front-backdirection with respect to the user's body and the HMD 1. The Y directioncorresponds to a right-left direction with respect to the user's bodyand the HMD 1. The Z direction corresponds to a up-down direction withrespect to the user's body and the HMD 1.

The head attached part 1A has, for example, a spectacle-shaped housing10, and components such as a display device including a transmissivetype display surface 5 and a camera 6 are mounted on the housing 10. Ata part of the housing 10, for example, near each of left and right sidesurfaces, a marker portion 13 for distance measurement is included, anda plurality of markers 17 serving as measurement points are formed inthe marker portion 13.

The neck attached part 1B has housings 11 and a housing 12 as, forexample, arc-shaped housings. The housings 11 are portions arranged atleft and right positions with respect to the user's neck and shoulders,and have a right-side housing 11R and a left-side housing 11L. Thehousing 12 is a portion that connects the left and right-side housings11 and that is arranged at a position behind the user's neck andshoulders. A distance measuring sensor 7, a microphone 8, a speaker 9,an operation input unit 14, and the like are mounted on each housing 11.

The HMD 1 has not a configuration in which states of a position and adirection etc. of an HMD are measured by a difference in displacementfrom an initial state like a conventional HMD but a configuration inwhich each positional relationship of a position and a direction etc. ofthe head attached part 1A is directly measured from the distancemeasuring sensor 7 of the neck attached part 1B. Consequently, the HMD 1can omit mounting a component such as a position/attitude sensor on thehead attached part 1A, and reduces weight of the head attached part 1A.

As sensors for measuring the position and the attitude in theconventional HMD, given are an acceleration sensor, an angular velocitysensor (gyro sensor), a geomagnetic sensor, a GPS receiver, a camera,and a distance measuring sensor, etc. Incidentally, these devicesincluding a camera may be collectively referred to as a sensor. Sincethe acceleration sensor can measure a moving amount of the HMD andmeasure the gravitational acceleration vector at a time ofmotionlessness, it can measure an inclination of the HMD. The angularvelocity sensor measures a change in directions of the HMD. Thegeomagnetic sensor estimates a direction of the HMD based on detectionof a direction of the geomagnetism. The GPS receiver can know a positionof the HMD as values of latitude and longitude based on informationreceivable by a GPS radio wave. The camera captures feature points ofthe outside world. The position and the direction of the HMD can beestimated from a direction, in which the feature points are located, andmap information. The distance measuring sensor measures a distance to afeature point in the outside world. The position and the direction ofthe HMD can be estimated from the distance and the map information.

In the HMD 1 according to the first embodiment, at least some of thesensors for measuring the states of the HMD as described above aremounted on not the head attached part 1A but the neck attached part 1B.Therefore, the HMD 1 has a function for measuring the relativelypositional relationship between the neck attached part 1B and the headattached part 1A. The HMD 1 uses the positional relationship to performconversion between data measured by the sensor of the neck attached part1B and data measured by the sensor of the head attached part 1A.

The HMD 1 measures a distance from the distance measuring sensor 7mounted on the neck attached part 1B to the housing 10 of the headattached part 1A in order to grasp the positional relationship. Inparticular, the distance measuring sensor 7 measures positions of aplurality of markers 17 in the housing 10 as a plurality of featurepoints. The plurality of feature points may be three or more points thatare not on the same straight line. The neck attached part 1B can obtaina positional relationship between a head coordinate system of the headattached part 1A and a neck coordinate system of the neck attached part1B based on the measured feature points (FIG. 5 described later).

By using the obtained positional relationship, the HMD 1 can correct andconvert the state detected by the sensor of the neck attached part 1B toand into a state with reference to the head coordinate system of thehead attached part 1A. Further, by using the obtained positionalrelationship, the HMD 1 can correct and convert the state detected bythe sensor of the head attached part 1A to and into a state withreference to the neck coordinate system of the neck attached part 1B.Therefore, the HMD 1 can grasp both the states of the head's positionand direction etc. with reference to the head coordinate system and thestates of the body trunk's position and direction etc. with reference tothe neck coordinate system. The HMD 1 can perform both display controlof a virtual object according to the head's direction and the like anddisplay control of a virtual object according to the body trunk'sdirection and the like by using data on those states.

[HMD]

FIG. 2 shows a functional block configuration of the HMD 1 according tothe first embodiment. The head attached part 1A includes a processor100, a memory 120, a communication interface unit 130, a power supplycircuit 141, a sub-battery 142, a display device 150, a camera 6, anoperation input unit 18, and the like, and they are connected to oneanother via a bus or the like.

The processor 100 is a sub-controller that controls the head attachedpart 1A. Data and information handled by the processor 100 and the likeare stored in the memory 120. The communication interface unit 130 is aportion such as a communication circuit that performs wiredcommunication with the neck attached part 1B via the connecting line 4.The power supply circuit 141 is a portion that receives power feedingfrom the neck attached part 1B via the connecting line 4, and chargesthe sub-battery 142. The sub-battery 142 supplies electric power to eachportion in the head attached part 1A. The mounting of the sub-battery142 may be omitted. In that case, power feeding is supplied from thepower supply circuit 141 to each portion. The display device 150displays a video image or an image of a virtual object or the like on aregion of a display surface 5 based on display data. The camera 6captures an image of circumference including a front of the headattached part 1A. The operation input unit 18 includes, for example, anoperation button or the like for operating the HMD 1. The mounting ofthe operation input unit 18 can be omitted. The marker portion 13includes a plurality of markers 17, and may include a circuit or thelike that controls light emission of the markers 17.

The neck attached part 1B includes a processor 200, a memory 220, acommunication interface unit 230, a power supply circuit 241, a mainbattery 242, an operation input unit 14, a distance measuring sensor 7,a position/attitude sensor 70, a voice input unit, a voice output unit,and the like, and they are connected to one another via a bus or thelike.

The processor 200 is a main controller that controls the entire HMD 1including the neck attached part 1B. Data and information handled by theprocessor 200 and the like are stored in the memory 220. Thecommunication interface unit 230 is a portion such as a communicationcircuit that performs wired communication with the head attached part 1Avia the connecting line 4. In addition, the communication interface unit230 also performs wireless communication with an outside and short-rangewireless communication with the operating tool 2. The power supplycircuit 241 is a portion that charges the main battery 242 from anexternal power source and also supplies power feeding to the neckattached part 1B through the connecting line 4. The main battery 242supplies electric power to each portion in the neck attached part 1B.The operation input unit 14 includes, for example, an operation buttonor the like for operating the HMD 1. The distance measuring sensor 7 isa combined type described later, and performs both distance measurementof the marker 17 and normal distance measurement. The position/attitudesensor 70 includes an acceleration sensor 71, an angular velocity sensor72, a geomagnetic sensor 73, and a GPS receiver 74. The audio input unitincludes the right and left microphones 8. The audio output unitincludes the right and left speakers 9 and earphones.

The memory 220 also stores a control program for configuring thefunctions of the HMD 1, an application program for realizing eachapplication, setting information, and the like. The setting informationincludes system setting information and user setting information. Thecontrol program or application program is, for example, a program thatdisplays a virtual object (corresponding display object) including agraphical user interface (GUI) for work support in a user's field ofview.

In the first embodiment, the neck attached part 1B has been described asa portion that is attached near the neck and shoulders of the user'sbody trunk, but the present embodiment is not limited to this. The neckattached part 1B may have such a configuration as to be able to detect astate such as a direction of the user's body trunk, so that it can be abody trunk attached part as a portion to be attached to any portion ofthe body trunk. That is, the body trunk attached part may be a portionto be attached to a chest, a back, a stomach, a waist, or the like.

The reason why the vicinity of the neck is selected as an attachedlocation in the first embodiment also has the following points besides apoint convenient to use of a speaker and a microphone. That is, thereare points in which a distance between the head attached part and thebody trunk attached part is shortened as much as possible and a lengthof a cable (corresponding connecting line 4) in connecting them by wireis shortened as much as possible. As the cable is shorter, a feeling ofthe cable clinging to the body can be made less and a user's wearingfeeling, ease of movement, and usability can be made better. As amodification example, connection between the head attached part 1A andthe neck attached part 1B regarding the communication and power feedingis not limited to wired connection, and may be wireless.

[HMD—Processing Unit]

FIG. 3 shows a functional block configuration of a control unit and thelike realized by a processor 100 of the head attached part 1A and aprocessor 200 of the neck attached part 1B.

The head attached part 1A includes a data acquisition unit 100 a, a dataprocessor 100 b, a communication controller 100 c, and a displaycontroller 100 d as respective processing units in the control unitrealized by the processor 100 or the like. The head attached part 1Astores display information D11 and the like in a storage unit realizedby a memory 120 and the like. The display information D11 is displaydata or the like received from the neck attached part 1B.

The neck attached part 1B includes a data acquisition unit 200 a, a dataprocessor 200 b, a communication controller 200 c, and a displaycontroller 200 d as respective processing units in the control unitrealized by the processor 200 or the like. The neck attached part 1Bstores display information D21, coordinate system information D22, andthe like in a storage unit realized by the memory 220 and the like. Thedisplay information D21 corresponds to display data including an imagefor display control and control information, and includes data to betransmitted to the head attached part 1A. The coordinate systeminformation D22 corresponds to information for managing and controllingeach coordinate system of a head coordinate system and a neck coordinatesystem, which will be described later, and their positionalrelationship.

The data acquisition unit 100 a of the head attached part 1A acquiresdata such as an image taken by the camera 6 and stores it in the storageunit. Incidentally, when the head attached part 1A is provided withanother sensor, the data acquisition unit 100 a acquires data detectedby the sensor and stores it in the storage unit. The data processor 100b processes the data acquired by the data acquisition unit 100 a asnecessary and sends it to the communication controller 100 c. Thecommunication controller 100 c controls transmission of the data to theneck attached part 1B via a communication interface unit 130.

The data acquisition unit 200 a of the neck attached part 1B acquiresdata detected by each sensor of the position/attitude sensor 70 and datadetected by normal distance measurement of the distance measuring sensor7, and stores them in the storage unit. The communication controller 200c receives, via the communication interface unit 230, the data from thecommunication controller 100 c of the head attached part 1A and sends itto the data processor 200 b. Further, the processor 200 uses thedistance measuring sensor 7 to cause it to measure a distance of themarker 17 of the head attached part 1A. The data acquisition unit 200 aalso acquires its distance measurement data. The data processor 200 bprocesses each piece of data acquired by the data acquisition unit 200 aand each piece of data from the head attached part 1A as necessary.

The data processor 200 b calculates a relatively positional relationshipwith the head attached part 1A with respect to the neck attached part 1Bbased on the distance measurement data of the marker 17. This positionalrelationship corresponds to a positional relationship and a rotationalrelationship of the origin between the neck coordinate system and thehead coordinate system. The data processor 200 b stores the calculateddata on the positional relationship in the coordinate system informationD22.

The data processor 200 b calculates states such as a position and adirection of the neck coordinate system and states such as a positionand a direction of the head coordinate system by using the positionalrelationship. At that time, the data processor 200 b corrects andconverts data detected by the position/attitude sensor 70 of the neckattached part 1B to and into data representing the states in the headcoordinate system based on the positional relationship. At that time,the data processor 200 b performs the correction and conversion based oncalculation of rotation of the coordinate system described later. Thedata processor 200 b stores, in the coordinate system information D22,data such as the states obtained by the calculation.

The display controller 200 d refers to data such as the states obtainedby the data processor 200 b, and controls display of a virtual objectonto the display surface 5. The display controller 200 d makes, forexample, display data for displaying an image at a position aligned withthe head direction and display data for displaying an image at aposition aligned with the body trunk direction, and stores them in thedisplay information D22. Further, the display controller 200 d mayperform display control that uses image data or the like of the camera 6of the head attached part 1A.

The communication controller 200 c controls transmission of the displaydata of the display information D21 to the head attached part 1A via thecommunication interface unit 230. The communication controller 100 c ofthe head attached part 1A receives the display data and stores it in thedisplay information D11. The display controller 100 d controls a displaydevice 150 according to the display data, and causes the display surface5 to display an image thereof.

Incidentally, in the first embodiment, since the display controller 100d is configured to display the image on the display surface 5, as it is,according to the display data from the display controller 200 d, themounting of the display controller 100 d can be omitted or simplified.In another embodiment, the display controller 100 d may perform uniquedisplay control on the head attached part 1A independently of thedisplay control by the display controller 200 d. For example, thedisplay controller 100 d may perform display control that uses animage(s) of the camera 6. Further, when the head attached part 1A has aline-of-sight detecting function, the display controller 100 d maycontrol image display according to a line-of-sight direction detected byusing the function. Further, if the head attached part 1A is configurednot to have the camera 6 or other sensors, the mounting of the dataprocessor 100 b can be omitted or simplified. A configuration as toperform a main processing(s) by a main controller of the neck attachedpart 1B makes it possible to reduce the mounting of the devices on thehead attached part 1A.

Further, in the first embodiment, the data processor 200 b of the neckattached part 1B performs calculation such as conversion of the statethat uses the positional relationship, but in another embodiment, thesame calculation may be performed by the data processor 100 b of thehead attached part 1A after transmitting the data on the positionalrelationship from neck attached part 1B to the head attached part 1A.

[HMD—Processing Flow]

FIG. 4 shows a main processing flow in the HMD 1 according to the firstembodiment. FIG. 4 has steps S11 to S17 by the neck attached part 1B andsteps S21 to S26 by the head attached part 1A. In step S11, when poweris turned on based on a user's operation, the neck attached part 1Bperforms a start-up processing to start a circuit operation andcommunicates with the head attached part 1A. In step S21, the headattached part 1A starts a circuit operation in response to communicationfrom the neck attached part 1B.

In step S12, the neck attached part 1B acquires each piece of data bythe data acquisition unit 200 a. The acquired data at this time includesdetected data by the position/attitude sensor 70 and distancemeasurement data on the distance measurement of the marker 17 by thedistance measuring sensor 7. In step S22, the head attached part 1Aacquires data by the data acquisition unit 100 a. In step S23, the headattached part 1A transmits the data to the neck attached part 1B by thecommunication controller 100 c. In step S13, the neck attached part 1Breceives the data by the communication controller 200 c.

In step S14, the data processor 200 b calculates a positionalrelationship with the head attached part 1A by using the acquired data,and performs correction and conversion about the states detected by theposition/attitude sensor 70 by using the positional relationship.Consequently, the data processor 200 b obtains both data representingthe states in the neck coordinate system and data representing thestates in the head coordinate system, and updates the coordinate systeminformation D22.

In step S15, the display controller 200 d makes display data for displaycontrol of a virtual object based on the coordinate system informationD22, and stores it in the display information D21. In step S16, thecommunication controller 200 c transmits the display data to the headattached part 1A. In step S24, the communication controller 100 c of thehead attached part 1A receives the display data. In step S25, thedisplay controller 100 d displays an image(s) on the display surface 5based on the display data.

In step S17, when the power is turned off (Yes) based on the user'soperation, the neck attached part 1B performs an ending processing ofthe neck attached part 1B and this flow is ended. At a time of theending processing, communication to the head attached part 1A is alsoperformed. If the power remains on (No), this processing returns to stepS12 and the same processing is repeated. Further, in step S26, whenpower of the neck attached part 1B is turned off (Yes), the headattached part 1A performs an ending processing of the head attached part1A and this flow ends. If the power remains on (No), this processingreturns to step S22 and the same processing is repeated.

[Coordinate System and Positional Relationship]

FIG. 5 shows a configuration related to each coordinate system and apositional relationship in the HMD 1 according to the first embodiment.Incidentally, FIG. 5 omits illustrations of the connecting line 4, theuser's body, and the like. (A) shows a first example of a state of theHMD1, and (B) shows a second example of the state. In the state of (A),a direction of the body trunk such as the user's neck and a direction ofthe head are almost the same, and this state is forward. Correspondinglythereto, a direction (axis X_(N)) of the neck attached part 1B and adirection (axis X_(H)) of the head attached part 1A are almost the same.It is assumed that a coordinate system of the head attached part 1A is ahead coordinate system CH {axis X_(H), Y_(H), Z_(H)}, and an origin isthe origin O_(H). It is assumed that a coordinate system of the neckattached part 1B is a neck coordinate system CN {axis X_(N), Y_(N),Z_(N)}, and an origin is the origin ON. A position of the origin ON isdefined based on a relatively positional relationship from each portion(particularly, a position of the distance measuring sensor 7) of theneck attached part 1B. In this state, axes X_(H) and X_(N) correspond toa forward direction; axes Y_(H) and Y_(N) correspond to a leftdirection; and axes Z_(H) and Z_(N) correspond to an upward direction.The neck coordinate system CN is, in other words, a body trunkcoordinate system.

A vector V1 is a vector representing a relatively positionalrelationship with the head attached part 1A with respect to the neckattached part 1B, and is a vector from the origin O_(N) to the originO_(H). Incidentally, when an opposite vector to the vector V1 isconsidered, the vector is a vector representing a relatively positionalrelationship with the neck attached part 1B with respect to the headattached part 1A.

The state of (B) is an example in which the user changes the directionand position of the head from the state of (A) without nearly changingthe body trunk. In the state of (B), the state of the neck attached part1B is almost unchanged from the state of (A). In the state of (B), theposition of the head attached part 1A has moved slightly forward withrespect to the state of (A), and the direction of the head attached part1A has rotated slightly on a left side.

The HMD 1 distance-measures a plurality of markers 17 of the headattached part 1A from the distance measuring sensor 7 of the neckattached part 1B. The vector v1 is a vector from the position of thedistance measuring sensor 7 to the position of the marker 17 at the timeof the distance measurement. The HMD 1 calculates a relativelypositional relationship between the neck attached part 1B and the headattached part 1A based on distance measurement data. This positionalrelationship is represented by the vector V1 between the origins and bythe rotation between the head coordinate system CH and the neckcoordinate system CN.

Measured points (corresponding markers 17) that require the distancemeasurement to calculate the positional relationship are at least threepoints that are not on the same straight line. An illustrated example isconfigured so that a total of four points are distance-measured on theleft and right sides by using the two distance measuring sensors 7 onthe left and right sides, but the present embodiment is not limited tothis. As a result, if the three points can be distance-measured, therelational relationship is calculable. In the illustrated example, thepositions of the markers 17 are one location of a front end on each sidesurface of the housing 10 and one location in each middle thereof, butthe present embodiment is not limited to this. Furthermore, in thisconfiguration example, each of the two right and left distance measuringsensors 7 distance-measures two points on the left and right sides ofthe housing 10. The present embodiment is not limited to this, and theleft and right markers 17 of the housing 10 may be distance-measuredfrom one distance measuring sensor 7.

Further, in FIG. 5, the distance measuring sensor 7 is a combined type,and can perform two types of distance measurement, that is, distancemeasurement 501 for calculating the positional relationship and normaldistance measurement 502. The distance measurement 501 is distancemeasurement of such a marker 17 as to include the vector v1. The normaldistance measurement 502 is distance measurement to a surrounding objectincluding a front direction of the HMD 1. The normal distancemeasurement 502 is, for example, distance measurement for grasping astructure of a surrounding room. The present embodiment is not limitedto this. In a modification example of the HMD, a distance measuringsensor for the distance measurement 501 for calculating the positionalrelationship and the distance measuring sensor for the normal distancemeasurement 502 may be provided separately. For example, the distancemeasuring sensor for the normal distance measurement 502 may be providedat a position on a front side of the housing 11, and the distancemeasuring sensor for the distance measurement 501 may be provided at aposition on a middle or a back side of the housing 11 or at the housing12. Furthermore, these two types of distance measuring sensors may beseparately mounted on the head attached part 1A and the neck attachedpart 1B.

[Distance Measuring Sensor]

A TOF sensor (TOF: Time of Flight), a stereo system camera, or the likecan be applied to the distance measuring sensor 7. The TOF sensordetects light returning after irradiated reference light has hit anobject, and calculates a distance from a flight time of the light. Inapplying the TOF sensor, a reflective member that efficiently reflectslight having a wavelength of the reference light may be arranged, as themarker 17, on the head attached part 1A. The wavelength of the referencelight is, for example, a wavelength in a near infrared region.Consequently, an intensity of the reflected light of the TOF sensor isincreased, a measurement error can be reduced, and the feature pointscan be easily detected. Furthermore, in using the reference light in thenear infrared region, the present embodiment also has such an effectthat an influence of ambient light outdoors becomes stronger.

The stereo system camera calculates a distance from a difference betweenthe feature points in the captured left and right images. In applyingthe stereo system camera, a light emitting marker may be arranged as themarker 17 of the head attached part 1A. The marker portion 13 maycontrol light emission of the marker 17. This light emitting markeremits near-infrared light, for example. Consequently, incident light inthe stereo system camera is increased, the measurement error can bereduced, the feature points can be easily detected, and the same effectas that of the TOF camera can be obtained.

Further, in measuring the distance from the distance measuring sensor 7of the neck part 1B to the marker 17 of the head attached part 1A, itmay be difficult to measure the distance due to some influence. In thatcase, the HMD 1 measures an arbitrary feature point in the outside worldother than the marker 17 by both the distance measuring sensor 7 on aneck attached part 1B side and the camera 6 or another sensor on a headattached part 1A side, may estimate a positional relationship based onmeasured data of both of them. As a modification example, both the neckattached part 1B and the head attached part 1A may be provided with thedistance measuring sensor 7 and, in that case, the positionalrelationship can be estimated by using distance-measured data by twoupper and lower distance measuring sensors 7.

By using the distance measuring sensors 7 in combination, the number andweight of devices to be mounted can be reduced. When the distancemeasuring sensor 7 is separated for normal distance measurement and forpositional-relationship distance measurement, the number and weight ofdevices to be mounted increase, but the present embodiment has such anadvantage that the optimum distance measuring sensor can be selectedaccording to the measurement distance.

[Positional Relationship Calculation and Display Control]

FIG. 6 shows a processing example for the above-mentioned positionalrelationship calculation and the like. In step S01, the neck attachedpart 1B obtains distance measurement data DM on the distance measurementof the marker 17 by the distance measuring sensor 7. In step S02, theneck attached part 1B calculates the positional relationship with thehead attached part 1A by using the distance measurement data DM, andobtains positional relationship data DL. In step S03, the neck attachedpart 1B obtains data such as sensor data SN on states such as a positionand a direction with reference to the neck coordinate system CN by theposition/attitude sensor 70. In step S04, the neck attached part 1B usesthe positional relationship data DL from the sensor data SN of the neckcoordinate system CN and obtains, by correction and conversionprocessings based on the rotation of the coordinate system, data such assensor data SH representing states such as a position and a directionwith reference to the head coordinate system CH.

In step S05, the neck attached part 1B performs a display controlprocessing using the sensor data SN of the neck coordinate system CN anda display control processing using the sensor data SH of the headcoordinate system CH. For example, the neck attached part 1B makesdisplay data for displaying a virtual object aligned with the headdirection and display data for displaying a virtual object aligned withthe body trunk direction, and transmits them to the head attached part1A.

In step S06, the head attached part 1A displays each image on thedisplay surface 5 based on the display data from the neck attached part1B. That is, the head attached part 1A displays an image at a positionaligned with the body trunk direction according to the display data withreference to the neck coordinate system CN, and also displays an imageat a position aligned with the head direction according to the displaydata with reference to the head coordinate system CH.

[Neck Attached Part]

FIG. 7 shows a mounting example of components in the neck attached part1B. The neck attached part 1B has a control circuit 701 and a battery702 mounted symmetrically inside each of left and right housings 11(11L, 11R) in the Y direction. In the housing 11, the battery 702 isarranged on a front side in the X direction, and the control circuit 701is arranged on a back side. The control circuit 701 corresponds to aportion such as an IC board on which each of components such as theprocessor 200, the memory 220, and the communication interface unit 230of FIG. 2 is mounted. The battery 702 corresponds to a portion on whichthe main battery 242 is mounted.

A certain component(s) is disposed at least at one location of the rightand left housings 11 or the housing 12 on its back side. Alternatively,a certain component may be divided into the left and right housings 11and arranged as a plurality of portions. Further, a certain componentmay be divided into the right and left housings 11 and duplicated. Forexample, the processor 200 is arranged in one of the right and leftcontrol circuits 701, and another component is arranged in the other ofthe control circuits 701. For example, the main battery 242 isseparately arranged and duplicated as two portions of the right and leftbatteries 702. One battery 702 may be actually used, and the otherbattery 702 may be used for charging. The right and left batteries 702may be configured to be interchangeable. In this case, the user canreplace the other battery 702 while using one battery 702 withoutturning off the power. Furthermore, for example, the distance measuringsensor 7 is separately arranged as two distance measuring sensors in theright and left housings 11. The distance measuring sensor 7 and othersensors may be duplicated so that only one of right and left sensors canfunction.

Like this example, a right and left weight balance of the entire HMD 1is taken by making its configuration symmetrical as much as possiblewith respect to the mounting of each component. That is, weight of theright-side housing 11R and weight of the left-side housing 11L are setto almost the same. In addition, the weight balance is taken in afront-back direction including the housing 12. Consequently, a wearingfeeling of the HMD 1 by the user is made better. In a case of aredundant configuration of duplicating a certain component like theright and left housings 11, an effect such as improvement ofavailability can be obtained. Incidentally, without being limited toarrangement shown in FIG. 7, for example, the control circuit 701 may bearranged on the front side and the battery 702 may be arranged on theback side in the X direction. Other components such as the microphone 8and the speaker 9 are also arranged in each of the right and lefthousings 11 so as to balance the weight on the right and left sides.

The power consumption of the HMD tends to increase as the functionbecomes more sophisticated. Battery capacity and weight required inmounting the battery also tend to increase according to the increase inthe power consumption. Therefore, in the example of the firstembodiment, a reduction in the weight of the head attached part 1A isachieved by mounting the main battery 242 on the neck attached part 1B.At the same time, the right and left weight balance of the neck attachedpart 1B is taken by symmetrically arranging parts having the same weightas the two batteries 702 in the right and left housings 11 of the neckattached part 1B. The right and left weight balance may be achieved byarranging the same components on the right and left sides, or may beestablished between a certain component (for example, the processor 200)and another component. Consequently, the weight balance of the entireHMD 1 can be easily taken, and a cause of positional displacement of theneck attached part 1B during the mounting is also reduced.

[Connecting Line]

FIG. 8 shows a configuration example about attachment or the like of theconnecting line 4 in the HMD 1. (A) is a first configuration example,which is a configuration example in consideration of ease of themounting. The HMD 1 has respective left and right connecting lines 4(4L, 4R). One end of the connecting line 4 is connected to a terminalnear each center of right and left parts of the housing 10 of the headattached part 1A and at each position in front of user's ears at a timeof being attached to the user. The other end of the connecting line 4 isconnected to a terminal located near each central position of the rightand left housings 11 of the neck attached part 1B. Each connecting line4 passes in front of the user's ear and becomes hanging down therefrom.An interval 801 is a distance between attachment positions of the leftand right connecting lines 4 (4L, 4R) of the housing 10 of the headattached part 1A in the Y direction. An interval 802 is a distancebetween attachment positions of the left and right connecting lines 4 ofthe housing 11 of the neck attached part 1B in the Y direction. In theHMD 1, a positional displacement of the display surface 5 with respectto the head affects display performance. Therefore, in this way, thepresent embodiment is configured to symmetrically connect the headattached part 1A and the neck attached part 1B by the two connectinglines 2. Consequently, the right and left weight balances are taken andthe cause of the positional displacement is reduced, which ispreferable.

When the user wears (attaches) the HMD 1, the neck attached part 1B ishung on his or her neck/shoulder and the head attached part 1A is hungright in front of the head. When the user does not use the image displayby the head attached part 1A at a time of rest or the like, the user canalso remove the head attached part 1A from the head and become loweringit in front of his or her chest. Incidentally, in this configurationexample, connection or non-connection of the connecting line 4 to theterminal of the housing by the user is possible, but the connecting line4 may be fixedly connected to the housing.

(B) is a second configuration example. One end of each of the left andright connecting lines 4 (4L, 4R) is connected at each position of backends of left and right parts of the housing 10 of the head attached part1A and at each position behind the user's ears at the time of beingattached to the user. The other end of the connecting line 4 isconnected at a position closer to each back side of the left and righthousings 11 of the neck attached part 1B. Each connecting line 4 passesbehind the user's ear and becomes hangs down therefrom. Each of aninterval 803 and an interval 804 also indicates a distance betweenattachment positions of the connecting wires 4 similarly to (A).

An aspect of the connecting line 4 is not limited to the example of FIG.8, and may be another aspect. In another aspect, the other end of theconnecting line 4 may be connected to the housing 12 behind the neckattached part 1B, or may be combined as one connecting line 4 withoutbeing divided into left and right connecting lines.

Regarding the interval between the connecting lines 4, the firstconfiguration example of (A) shows a case where the interval 801 and theinterval 802 are substantially the same. The second configurationexample of (B) shows a case where the interval 804 is slightly widerthan the interval 803.

Various aspects are possible about the configurations of the housing ofthe HMD 1, the interval between the connecting lines 4, and the like.The interval 801 on the head side and the interval 802 on the neck sidemay be the same. The interval 802 on the neck side may be made largerthan the interval 801 on the head side. The interval 802 on the neckside may be made smaller than the interval 801 on the head side.Further, the above-mentioned interval is fixedly designed in advance,but as another form, for example, the user may be able to adjust theabove-mentioned interval by configuring the housing so as to be soft andflexible or deformable. For example, in the neck attached part 1B, theleft and right housings 11 may be configured by rigid members, and theback housing 12 may be configured by bending members. The user canadjust the interval 802 of the neck attached part 1B according to aphysical size, clothes, a wearing feeling, and the like. Furthermore, inadjusting the interval 802, the HMD 1 may measure the interval 802 atthat time by using, for example, the distance measuring sensor 7 oranother sensor. The intervals 801, 802 and a positional relationshipbetween components such as various sensors arranged in the housing ofthe HMD1 are preset as setting values in order to realize the functionsof the HMD 1. When the interval 802 or the like is changed, theabove-mentioned setting value is also updated.

A shape of the neck attached part 1B is not limited to an arc shape, andmay have another shape. It may be, for example, such a shape that afront neck of the housing 11 is closed. Each of the left and righthousings 11 has a rod shape, but is not limited to this, and may have ashape close to flat like a shoulder pad. The neck attached part 1B mayhave a clothing shape like, for example, a vest or work clothing.

FIG. 9 shows a configuration example of a distance between the left andright parts of the housing of the HMD 1 in the Y direction in relationto FIG. 8. FIG. 9 schematically shows a front face of a state where theuser has attached the head attached 1A and the neck attached part 1B. Awidth W1 indicates a width between the left and right parts of thehousing 10 of the head attached part 1A in the Y direction. A width W2indicates a width between the left and right parts of the housing 11 ofthe neck attached part 1B in the Y direction. (A) shows a case where thewidth W1 and the width W2 are almost the same. (B) shows a case wherethe width W2 is larger than the width W1 (W1<W2).

The HMD 1 desirably has a configuration in which the positions of thedistance measuring sensor 7 and the marker 17, etc. and dimension of thewidth etc. of each part are designed so that the distance measurementfrom the distance measuring sensor 7 of the neck attached part 1B to themarker 17 of the head attached part 1A is easily performed. In a case of(A), the marker 17 is measured upward from the distance measuring sensor7. In a case of (B), the marker 17 is measured in a slightly obliquedirection above the distance measuring sensor 7. In the firstembodiment, the width W2 on the neck side is set to the width W1 or moreon the head side in consideration of ease of the mounting, a reductionin factors of the positional displacement, ease of the distancemeasurement, and the like (W1≤W2). The present embodiment is not limitedto this, and can also adopt another setting. In other forms, the widthW2 on the neck side may be smaller (W1>W2).

Incidentally, the marker 17 may be arranged on a downward surface of thehousing 10, or may be arranged on a lateral surface. The distancemeasuring sensor 7 may be arranged on an upward surface of the housing11, or may be arranged on a lateral surface. As a modification example,in (A), a portion such as a distance measuring sensor 7 or anothersensor may be arranged at each position 901 which extends to the leftand right outside from the housing 11 of the neck attached part 1B andis separate from the housing. Further, the modification example may havesuch a form that a length of the extension can be adjusted by the user.

[Separation Configuration Example]

FIG. 10 shows a table that summarizes some configuration examples inwhich the components are divided into upper and lower portions of thehead attached part 1A and the neck attached part 1B and are mounted inthe HMD 1 of the first embodiment and each modification example.

First, the configuration in the first embodiment is shown in FIG. 2 andthe like described above, but the neck attached part 1B is provided witha distance measuring sensor 7, a position/attitude sensor 70, a mainbattery 242, a microphone 8, a speaker 9, and the like. The headattached part 1A is provided with a camera 6, a display device 150, asub-battery 142, and the like. The position/attitude sensor 70 includesthe above-mentioned acceleration sensor 71, angular velocity sensor 72,geomagnetic sensor 73, and GPS receiver 74, and the distance measuringsensor 7 and the camera 6 are considered separately. Since the neckattached part 1B is fixed to the user's body trunk, theposition/attitude sensor 70 detects a state of a direction or the likeof the user's body trunk. In normal distance measurement, the distancemeasuring sensor 7 measures a distance to an object around the neckattached part 1B (FIG. 5). In photographing the outside world with thecamera 6, it is advantageous that the outside world close to the user'sviewpoint is more easily photographed. Therefore, in the firstembodiment, the camera 6 is provided on the head attached part 1Alocated at a position higher than that of the neck attached part 1B.

Next, a first modification example of the first embodiment has aconfiguration different from that of the first embodiment in that thecamera 6 is provided to the neck attached part 1B. This makes itpossible to further reduce the weight of the head attached part 1A.Incidentally, in the first modification example, the image of the camera6 is an image with reference to the neck coordinate system CN.Therefore, when it is desired to obtain image information with referenceto the head coordinate system CH, the image of the camera 6 may becorrected and converted by using the above-mentioned positionalrelationship. The first modification example emphasizes weight reductionof the head attached part 1A, and many components are mounted on theneck attached part 1B.

Next, a second modification example has a configuration different fromthat of the first embodiment in that the distance measuring sensor fornormal distance measurement is provided, as a second distance measuringsensor, to the head attached part 1A separately from the distancemeasuring sensor 7 of the neck attached part 1B. Since more easilyseeing the outside world is advantageous in distance-measuring anobject(s) in the outside world, the second modification example providesthe second distance measuring sensor to the head attached part 1A whichis arranged at a higher position.

Next, a third modification example has a configuration different fromthat of the first embodiment in that one partial sensor of theposition/attitude sensor 70 is provided in the head attached part 1A andthe other partial sensor thereof is provided in the neck attached part1A. In particular, the acceleration sensor 71 and the angular velocitysensor 72 are mounted on the head attached part 1A, and the geomagneticsensor 73 and the GPS receiver 74 are mounted on the neck attached part1B. Since accuracy of detecting a rotational direction and a gravitydirection of the head is particularly important, it is an advantage thatthe sensor is mounted on the head attached part 1A similarly to thethird modification example when the advanced function is emphasized.

Next, a fourth modification example is provided with a position/attitudesensor, a type whose overlaps, to both the neck attached part 1B and thehead attached part 1A. In particular, a high-performance sensor ismounted as a main sensor on the neck attached part 1B, and a lightweightsensor is mounted as a sub-sensor on the head attached part 1A. Forexample, the HMD 1 normally acquires high-precision data by using thesensor of the neck attached part 1B, and may acquire sensor data of thehead attached part 1A if the sensor data of the neck attached part 1Bcannot be acquired for any reason. The fourth modification example has aconfiguration in which high accuracy and availability are emphasizedwhile the weight of the head attached part 1A is suppressed.

As another modification example, the microphone 8, the speaker 9 and thelike may be provided to the head attached part 1A instead of the neckattached part 1B. Of course, a form of combining various configurationsas described above can be adopted.

[Coordinate System]

A relationship between the head coordinate system CH of the headattached part 1A and the neck coordinate system CN of the neck attachedpart 1B in FIG. 5 will be described below. Incidentally, in thefollowing explanation, the coordinate systems are unified to theright-handed system. Rotation of the coordinate system between the headcoordinate system CH and the neck coordinate system CN can be expressedby Euler angles or normalized quaternions. In the following, anormalized quaternion is used to represent the rotation of thecoordinate system. The processor 200 (particularly, the data processor200 b of FIG. 3) performs calculation including such rotation of thecoordinate system.

A normalized quaternion is a quaternion with a norm of 1 and canrepresent rotation around a certain axis. A normalized quaternion qrepresenting rotation of an angle η by using a unit vector (n_(X),n_(Y), n_(Z)) as a rotational axis is given by Equation 1 as describedbelow.

q=cos(η/2)+n _(X) sin(η/2)i+n _(Y) sin(η/2)j+n _(Z) sin(η/2)k  Equation1:

Here, i, j, and k are units of quaternions. Clockwise (right-hand)rotation in a case of facing a direction of the vector (n_(X), n_(Y),n_(Z)) is rotational direction in a case where η is positive. Rotationof any coordinate system can be represented by this normalizedquaternion.

Usage of symbols will be summarized as follows. The real part of thequaternion q is represented by Sc(q). The conjugate quaternion of thequaternion q is represented by q*. An operator that normalizes the normof the quaternion q to 1 is defined by [⋅]. Assuming that q is anarbitrary quaternion, Equation 2 as described below is definition of theoperator [⋅]. The denominator on the right side of Equation 2 is thenorm of the quaternion q.

[q]=q/(q q*)^(1/2)  Equation 2:

Next, a quaternion p representing a coordinate point or a vector (p_(X),p_(Y), p_(Z)) is defined by Equation 3 as described below.

p=p _(X) i+p _(Y) j+p _(Z) k  Equation 3:

In the present specification, unless otherwise specified, it is assumedthat symbols representing coordinate points and vectors which are notdisplayed as components are displayed as quaternions. Further, it isassumed that a symbol representing rotation is a normalized quaternion.

It is assumed that PT(n) is a projection operator of a vector onto aplane perpendicular to a direction of a unit vector n. A projection of avector p is expressed by Equation 4 as described below.

P _(T)(n)p=p+nSc(np)  Equation 4:

It is assumed that a coordinate point or a directional vector p₁ isconverted into a coordinate point or a directional vector p₂ by arotational operation of an origin center represented by the quaternionq. By doing so, the directional vector p₂ can be calculated by Equation5 as described below.

p ₂ =qp ₁ q*  Equation 5:

It is assumed that a normalized quaternion obtained by rotating aroundan axis perpendicular to a plane including n₁ and n₂ so that a unitvector n₁ is superimposed on a unit vector n₂ is R (n₁, n₂). R (n₁, n₂)is given by Equation 6 as described below.

R(n ₁ ,n ₂)=[1−n ₂ n ₁]  Equation 6:

Next, a relationship between the head coordinate system CH of the headattached part 1A and the neck coordinate system CN of the neck attachedpart 1B will be described. When the user moves his/her neck or head, apositional relationship between the two coordinate systems changes. Forexample, a state of (A) in FIG. 5 changes to a state of (B). In the HMD1 of the first embodiment, a positional relationship between the twocoordinate systems is grasped by constantly measuring the featurepoints, which are indicated by the markers 17 of the head attached part1A, from the distance measuring sensor 7 of the neck attached part 1B.

Coordinates of a measurement point (corresponding marker 17) in the headcoordinate system CH, in other words, a shape or the like of the housing10 is known in advance by design. It is assumed that coordinate valuesof these three points in the head coordinate system CH are p_(H0),p_(H1), and p_(H2). Coordinate values of the three points in the neckcoordinate system CN can be obtained by the measurement by the distancemeasuring sensor 7. It is assumed that coordinate values of these threepoints are p_(N0), P_(N1), and p_(N2).

The HMD 1 first calculates rotation to align the direction of thecoordinate system. It is assumed that three measurement points are afirst measurement point, a second measurement point, and a thirdmeasurement point. It is assumed that representations of unit vectors inrespective directions directed from the first measurement point to thesecond measurement point and the third measurement point in the headcoordinate system CH are defined as n_(H1) and n_(H2). Similarly, it isassumed that representations in the neck coordinate system CN aredefined as n_(N1) and n_(N2). Specifically, these are given by Equation7 as described below. Incidentally, two directional vectors havingdifferent directions have only to be obtained, and the number ofmeasurement points is not limited to three.

n _(H1)=[p _(H1) −p _(H0)]

n _(H2)=[p _(H2) −p _(H0)]

n _(N1)=[p _(N1) −p _(N0)]

n _(N2)=[p _(N2) −p _(N0)]  Equation 7:

First, in the rotation in the representation of the head coordinatesystem CH, rotation q_(TA), which is rotation for superimposing n_(N1)on n_(N1), will be considered. Here, the rotation q_(TA) is given byEquation 8 as described below.

q _(TA) =R(n _(H1) ,n _(N1))  Equation 8:

Next, it is assumed that directions of rotating n_(N1) and n_(H2) due tothe rotation q_(TA) are defined as n_(A1) and n_(A2). This direction isgiven by Equation 9 as described below.

n _(A1) =q _(TA) n _(H1) q _(TA) *=n _(N1)

n _(A2) =q _(TA) n _(H2) q _(TA)*  Equation 9:

These are an angle between the same directions, so that an angle formedby n_(A1) and n_(A2) is equal to an angle formed by n_(N1) and n_(N2).Further, since the three measurement points are not on the same straightline, the angle formed by n_(N1) and n_(N2) is not 0 (zero). Therefore,n_(A1), that is, rotation q_(TB) obtained by using n_(N1) as an axis isuniquely determined, and n_(A2) can be superimposed on n_(N2).Specifically, the rotation q_(TB) is given by Equation 10 as describedbelow.

q _(TB) =R([P _(T)(n _(N1))n _(A2)],[P _(T)(n _(N1))n _(N2)])  Equation10:

By this rotation q_(TB), n_(A1) and n_(A2) are rotated to n_(N1) andn_(N2) of Equation 11 as described below.

n _(N1) =q _(TB) n _(A1) q _(TB)*

n _(N2) =q _(TB) n _(A2) q _(TB)*  Equation 11:

The rotation q_(T) is defined again by Equation 12 as described below.This rotation q_(T) is rotation in the head coordinate system CH, whichaligns the direction of the neck coordinate system CN with the directionof the head coordinate system CH.

q _(T) =q _(TB) q _(TA)  Equation 12:

Finally, a relationship between coordinate origins is obtained. If it isassumed that a coordinate origin of the neck coordinate system CN in thehead coordinate system CH is O_(NH), O_(NH) is given by Equation 13 asdescribed below.

O _(NH) =p _(H0) −q _(T) *p _(N0) q _(T)  Equation 13:

From the above, if positions of three or more feature points(corresponding markers 17), which are not on the same straight lineamong the feature points of the head attached part 1A, can be measuredfrom the neck attached part 1B, a relationship between the directions ofthe neck coordinate system CN and the head coordinate system CH, thatis, the above-mentioned positional relationship can be obtained. Thismakes it possible for the HMD 1 to use the head coordinate system CH andthe neck coordinate system CN in an integrated manner.

If the relationship between the head coordinate system CH and the neckcoordinate system CN calculated as described above is known, mutualconversion from a value of one coordinate system to a value of the othercoordinate system can be performed regardless of whether a sensor suchas the position/attitude sensor is arranged in the head attached part 1Aor the neck attached part 1B. Therefore, various devices including theabove-mentioned position/attitude sensor 70, specifically theacceleration sensor 71, angular velocity sensor 72, geomagnetic sensor73, GPS receiver 74, camera 6, and distance measuring sensor 7 may bebasically mounted on either the head attached part 1A or the neckattached part 1B. The HMD 1 of the first embodiment has a configurationin which at least some of the above-mentioned devices are mounted on notthe head attached part 1A but the neck attached part 1B. This makes itpossible to provide the advanced functions using the above-mentioneddevices and reduce the weight of the head attached part 1A.

[Display Information and Coordinate System Information]

FIG. 11 shows a configuration example of display information D21 andcoordinate system information D22 in the neck attached part 1B. A DB(database) of the display information D21 includes a display objectinformation table T01. In the display object information table T01,information such as an object ID, a display coordinate system,arrangement coordinates, and an arrangement direction is stored for eachdisplay object that is displayed on the display surface 5. The displaycoordinate system can be selected from each coordinate system. Thearrangement coordinates and the arrangement direction are positioncoordinates and a direction in the display coordinate system.

The DB of the coordinate system information D22 includes a coordinatesystem information table T02. The coordinate system information tableT02 stores information such as an origin position and a front directionfor each of the various coordinate systems. The various coordinatesystems include a world coordinate system (referred to as CW), a headcoordinate system CH, a neck coordinate system CN, an inertialcoordinate system (referred to as CI) described later, and the like.Origins of respective coordinate systems are set as an origin G1 of theworld coordinate system CW, an origin G2 (=O_(H)) of the head coordinatesystem CH, an origin G3 (=O_(N)) of the neck coordinate system CN, andan origin G4 of the inertial coordinate system CI, and they haveposition coordinates with reference to a certain coordinate system. Thefront direction of each coordinate system is represented, for example,as a direction of a certain axis.

[Effects etc.]

As described above, the HMD 1 of the first embodiment makes it possibleto realize a good balance between advanced functions and weightreduction by a separable type configuration and to improve the usabilityof the user.

As a modification example, when a configuration of providing the camera6 on the neck attached part 1B side (modification example 1 in FIG. 10)is adopted, a wide-angle camera may be applied as the camera 6. Thewide-angle camera is, for example, a camera capable of taking an imageat a field angle of 180 degrees or more. The HMD 1 determines and cuts apart out of a wide-angle image taken by the wide-angle camera, the partof the wide-angle image corresponding to the head direction of the headattached part 1A, and may use the cut image as image display controlwith reference to the head direction. This configuration is particularlyeffective in a case of the non-transparent display surface 5. Further,the wide-angle camera may be used in combination for normallyphotographing the outside world and for measuring the positionalrelationship.

Second Embodiment

An HMD according to a second embodiment of the present invention will bedescribed with reference to FIG. 12. Hereinafter, components differentfrom those of the first embodiment in a second embodiment and the likewill be described. In an HMD 1 according to a second embodiment, thedistance measuring sensor 7 is provided to the head attached part 1A,and a plurality of markers 17 are provided on the neck attached part 1B.That is, the distance measurement of the positional relationship betweenthe neck attached part 1B and the head attached part 1A in the secondembodiment has an opposite relationship to the first embodiment. The HMD1 distance-measures, as feature points, the plurality of markers 17 ofthe neck attached part 1B from the distance measuring sensor 7 of thehead attached part 1A, and calculates the positional relationship of theneck attached part 1B with respect to the head attached part 1A based onthe distance measurement data. The HMD 1 uses the positionalrelationship to perform correction and conversion between the data ofthe state detected by the sensor of the head attached part 1A and thedata of the state detected by the sensor of the neck attached part 1B.This calculation can be realized in the same manner as that in the firstembodiment.

FIG. 12 shows a configuration of an HMD 1 according to a secondembodiment. A distance measuring sensor 7, a camera 6, a display device150, and the like are mounted on the head attached part 1A. Aposition/attitude sensor 70, a battery (main battery 242), a microphone8, a speaker 9, and the like are mounted on the neck attached part 1B.In this example, the distance measuring sensor 7 of the head attachedpart 1A uses a combined type for normal distance measurement and fordistance measurement of a positional relationship. The distancemeasuring sensor 7 is symmetrically arranged as two distance measuringsensors near right and left ends and on the front side of the housing10, and distance-measures the plurality of markers 17 which aresubstantially below. Furthermore, in this example, three markers on eachof the left and right housings 11 of the neck attached part 1B, that is,a total of six markers are arranged. The vector V2 is a vectorrepresenting the positional relationship to the origin O_(N) of the neckcoordinate system CN seen from the origin O_(H) of the head coordinatesystem CH.

According to the HMD 1 of the second embodiment, the same effect as thatof the first embodiment can be realized. However, from the viewpoint ofthe weight of the head attached part 1A, it is more advantageous toprovide the distance measuring sensor 7 on the neck attached part 1Bsimilarly to the first embodiment. As a modification example of thesecond embodiment, the head attached part 1A may be provided, as aseparate body, with a second distance measuring sensor for normaldistance measurement. The second distance measuring sensor may beprovided on the neck attached part 1B. Further, as a modificationexample, the camera 6 may be mounted on the neck attached part 1B side.

Third Embodiment

An HMD according to a third embodiment will be described with referenceto FIG. 13. Due to an influence of user's clothes and the like, thepositional relationship between the head attached part 1A and the neckattached part 1B may not be optically measured by using the distancemeasuring sensor 7. In the third embodiment, a method capable ofmeasuring the positional relationship is shown even in such a case.

FIG. 13 shows a configuration of an HMD 1 according to a thirdembodiment. The HMD 1 is different from that of the first embodiment ina configuration point in which a position/attitude sensor 70 has sensorsincluding an acceleration sensor 71 and a geomagnetic sensor 73 in thehead attached part 1A and the neck attached part 1B.

First, a rotationally central position of the head attached part 1A asseen from the neck attached part 1B is determined according to anindividual and is substantially constant, so that this is taken as afixed position. A position of the HMD 1 is set in advance.Alternatively, the HMD 1 is set to a value at a time when the distancemeasuring sensor 7 or the like can optically measure its averagerotationally central position. For example, if a rotational center ofthe head attached part 1A is set as the coordinate origin of the headcoordinate system CH, a relationship between the coordinate systems isdetermined by obtaining a directional relationship between thecoordinate systems. Next, a relationship between the directions of therespective coordinate systems is obtained. Since the gravitationalacceleration can be measured by a 3-axis acceleration sensor which isthe acceleration sensor 71, a relationship between directions in thevertical direction can be obtained from a difference between measuredvalues of the respective coordinate systems in a gravitationalacceleration direction. Then, regarding a difference between directionsof horizontal planes of the head (corresponding head coordinate systemCH) and the body trunk (corresponding neck coordinate system CN), thegeomagnetic sensors 73 provided in both the head attached part 1A andthe neck attached part 1B are used, thereby being able to determine adifference in an azimuth direction between both horizontal planes. Thegeomagnetic sensor may bring occurrence of a deviation from an absolutevalue of the azimuth direction due to an influence of the outside world,but can obtain relatively stably a relative value. From the above, thedirectional relationship between the head coordinate system CH and theneck coordinate system CN can be calculated in the same manner as thatof the first embodiment.

As described above, in the third embodiment, a gravity accelerationdirection in each coordinate system and a direction in geomagnetism, forexample, the north direction, are used to calculate the directionalrelationship between the coordinate systems instead of the respectivedirections from the first measurement point toward the secondmeasurement point and the third measurement point regarding the threemeasurement points described in the first embodiment. Consequently, inthe third embodiment, even in a situation where the positionalrelationship between the head attached part 1A and the neck attachedpart 1B cannot be optically measured, the positional relationship can beestimated.

As described above, according to the third embodiment, the same effectas that of the first embodiment can be obtained from the viewpoint ofemphasizing the advanced functions. As a modification example of thethird embodiment, it may be configured not to include the distancemeasuring sensor 7.

The following is also possible as another modification example. The HMD1 may be in a situation where the positional relationship between theneck attached part 1B and the head attached part 1A cannot betemporarily measured. In that case, since the HMD 1 cannot performconversion using the positional relationship, it may be unable todisplay an image with reference to the neck coordinate system CN or animage with reference to the head coordinate system CH. In this case, asan exceptional handling processing, the HMD 1 may perform display thatuses no positional relationship, for example, display of an image withreference to the head coordinate system CH or an image with reference tothe neck coordinate system CN. Thereafter, when the positionalrelationship can be measured again, the HMD 1 stops the exceptionalhandling processing and returns to the original display.

Fourth Embodiment

A fourth embodiment will be described with reference to FIG. 14. An HMDaccording to a fourth embodiment has a configuration of mounting awireless power-feeding function about power feeding between the headattached part 1A and the neck attached part 1B and performing wirelesscommunication between the head attached part 1A and the neck attachedpart 1B.

FIG. 14 shows a configuration example of the HMD 1 according to thefourth embodiment. The connecting line 4 is composed of one line, andhas a loop shape by connection with the housing 10 of the head attachedpart 1A. One end of the connecting line 4 is connected to, for example,a back end on a left-side portion of the housing 10 of the head attachedpart 1A, and is connected to the power supply circuit 141 (FIG. 2) inthe housing 10. The other end of the connecting line 4 is connected to,for example, a back end of a right-side portion of the housing 10. Thispower supply circuit 141 includes a power reception circuit for wirelesspower feeding.

A middle portion of the connecting line 4 is a portion placed in contactwith the housing 12 of the neck attached part 1B, and mounts a powerreception antenna portion 4C. A power transmission antenna unit 12C ismounted on the housing 12. The power transmission antenna unit 12C isconnected to the power supply circuit 241 of the housing 11. The powersupply circuit 241 includes a power transmission circuit for wirelesspower feeding. For example, coils are formed in the power transmissionantenna unit 12C and the power reception antenna unit 4C, respectively.Wireless power feeding is performed between the power transmissionantenna unit 12C and the power reception antenna unit 4C, which are inclose proximity to each other, by an action of electromagneticinduction. Incidentally, a method of wireless power feeding is notlimited.

A wireless communication circuit 10C corresponding to a wirelesscommunication interface is mounted on, for example, the right-side partof the housing 10 of the head attached part 1A. A wireless communicationcircuit 11C is mounted on a part of the housing 11 of the neck attachedpart 1B, for example, a part of the right-side housing 11R. The HMD 1realizes a function in which upper and lower portions cooperate witheach other by appropriately performing wireless communication betweenthe wireless communication circuit 10C of the head attached part 1A andthe wireless communication circuit 11C of the neck attached part 1B.Incidentally, an antenna for wireless communication may be formed on orin the connecting line 4 or the housing 12.

Incidentally, a connection portion between the connecting line 4 and thehousing 12 may be configured to be separated and fixed by the user.Since the connecting line 4 is flexible and a certain length or morethereof is ensured as a margin, the user can freely move his/her neckand head.

Fifth Embodiment

A fifth embodiment will be described with reference to FIG. 15. An HMDaccording to a fifth embodiment has a function related to theabove-mentioned display control, and shows examples of display controlof images corresponding to a state of the head and a state of the bodytrunk of the user.

[Display Control]

An HMD of a conventional example uses, as a display method of an imageof a virtual object or the like, two standards of a world coordinatesystem of the outside world and a local coordinate system (headcoordinate system CH in the first embodiment) aligned with the head.Alternatively, like an example of Patent Document 1, proposed is alsouse of two standards of the coordinate system aligned with the head andthe coordinate system aligned with the body trunk (the neck coordinatesystem CN in the first embodiment). In contrast, the HMD1 of the fifthembodiment uses all three standards of the world coordinate system CW,the head coordinate system CH, and the neck coordinate system CN tocontrol the image display in each coordinate system. Consequently, theHMD 1 realizes the advanced functions and improves the user convenience.

Incidentally, in a method of the non-transparent type display surface 5,there is also an immersive method in which the entire visual field ofthe user is a display for displaying a VR image. Even in that case, inan aspect in which the user moves a “virtual world” in the image, if the“virtual world” is interpreted as a “world” referred to in the presentinvention, the same display method can be configured and the same effectis also obtained. Therefore, the display method of the present inventionincludes both a transparent type and a non-transparent type(particularly an immersive type) unless otherwise specified. If a caseof the immersive method is more specifically detailed, movement in the“virtual world” may be performed by a case of the actual movement of theHMD in the outside world or by a case of a control input by thecontroller (operating tool 2). Both can be regarded as movementsintended by the user within the world.

In the HMD 1 of the fifth embodiment, the following three types ofcontrol are used in combination about display control including controlof a position where an image is displayed on the display surface 5.

(1) Image display position control with reference to the worldcoordinate system CW. This display control is used in, for example, acase or the like of causing a virtual object for work support to bedisplayed at a position near an object in the outside world.(2) Image display position control with reference to the head coordinatesystem CH. This display control is used in, for example, a case ofcausing images such as system information, menus, commands, and icons inthe graphical user interface (GUI) of the HMD 1 to be displayed within afield of view corresponding to the user's head direction.(3) Image display position control with reference to the neck coordinatesystem CN. Meanwhile, regarding the display of a certain type of virtualobject, the display in the world coordinate system CW or the display inthe head coordinate system CH may not be convenient. For example,display of work procedures and work tools, etc. can be mentioned. Whenthis kind of virtual object is arranged at a fixed position withreference to, for example, the world coordinate system CW, it takes timeand effort to rearrange the virtual object each time the user moves.Incidentally, the movement of the user here includes not only themovement of the user in the real space but also the movement of theuser's viewpoint in the virtual space by the controller (operating tool2) or the like.

Therefore, in the fifth embodiment, the arrangement with reference tothe neck coordinate system CN is used for the display control of thiskind of virtual object. That is, the HMD 1 arranges this kind of virtualobject at a position aligned with the body trunk direction of the user.In a case of this arrangement, the virtual object is also displayed at aposition following the movement of the user, so that time and effort forthe rearrangement can be saved.

Meanwhile, in placing the above-mentioned virtual object at a positionwith reference to the head coordinate system CH, it is displayedfollowing the movement of the user, but it is placed at a fixed positionof a front visual field of the user, so that it may be difficult to seethe front visual field. For example, when the virtual object is placedat a central position of the front visual field, it becomes difficult tosee a work target(s) and work's hand. When the virtual object is placedat a position outside the central position of the front visual field,there is a limitation in that the virtual object becomes difficult tosee or that a large-sized virtual object or a large number of virtualobjects cannot be placed.

Therefore, in the fifth embodiment, display control with reference tothe neck coordinate system CN is used for this kind of virtual object.That is, in this display control, a target virtual object is displayedso as to follow intentional movement of the user in the world coordinatesystem CW and match it with the body trunk direction of the neckattached part 1B and so as not to follow a change of the head direction.In this display control, the target virtual object is displayed so thata display position and direction with respect to the display surface 5change according to the change of the head direction of the headattached part 1A with respect to the body trunk direction of the neckattached part 1B. This brings no limitation as mentioned above, andobtainment of an effect such as easier work. For example, the user canplace a virtual object such as a work tool at a position avoiding afront direction where an actual work object(s) is located, and the usercan place the large-sized virtual object or a large number of virtualobjects in the field of view of the above-mentioned virtual object withno problem if the head direction is changed.

[Display Control Example (1)]

FIGS. 15 to 18 show display control examples with reference to therespective coordinate systems. First, (A) of FIG. 15 shows an example ofthe entire visual field 5A of the user who wears (attaches) the HMD 1. AFOV (field of view) 5B exists at a center of the entire visual field 5A,and a work object 5D on a work table 5C exists as an example of a realthing. The entire visual field 5A is the entire range which the user canvisually recognize. The entire visual field 5A shows the entire visualfield at a time when the user is standing toward the work target 5D andnaturally faces the body trunk and his/her face toward a front. The FOV5B shows a range or a display region in which a display object can bedisplayed on the display surface 5 of the transparent type HMD 1. Apoint P1 shows a central point of the FOV 5B. (A) is a state in whichthe work object 5D and the like can be seen through the display surface5 from the user's point of view.

Incidentally, in a case of the non-transparent type HMD, the displayobject can be displayed by regarding the entire visual field 5A as adisplay region, and the FOV 5B in that case corresponds to a guide of aneasily visibly recognized range in front of the user's head and botheyes. In a case of the non-transparent type display surface 5, the workobject 5D or the like is displayed as a virtual object corresponding toa real thing by video see-through or simulation.

(B) shows an example of superimposedly displaying a display object suchas a virtual object on the FOV 5B of the display surface 5 with respectto the entire visual field 5A of (A), and is set to a first state. Anelement(s) other than the real thing is a display object (correspondingimage) such as a virtual object regardless of whether the HMD adopts amethod of a transparent type or a non-transparent type. In this example,the display objects include system information 61, an HMD menu 62, awork description 63, and a work tool 64.

The system information 61 is information provided to the user by asystem of the HMD 1 and is, for example, images representing states suchas a radio field intensity, time, and a remaining battery level.Further, the HMD menu 62 is a GUI component for the user to input aninstruction or the like and is a virtual object such as a HOME button.The system information 61 and the HMD menu 62 are examples of thedisplay with reference to the head coordinate system CH, and aredisplayed at fixed positions in the FOV 5B. For example, the systeminformation 61 is displayed in an upper-side area of the FOV 5B, and theHOME button, which is the HMD menu 62, is displayed in a lower-left areaof the FOV 5B. The system information 61 and the like are displayed atthe same positions in the entire visual field 5A and the FOV 5B evenwhen the position and the direction of the HMD 1 are changed due to themovement or the like of the user.

The work description 63 is a virtual object for explaining work aboutthe work object 5D to the user. The work description 63 is an example ofdisplay with reference to the world coordinate system CW, and isdisplayed at a fixed position near the work object 5D. The workdescription 63 is displayed at the same position in the world coordinatesystem CW even when the position and the direction of the HMD 1 changesdue to the movement or the like of the user.

The work tool 64 is a virtual object such as a tool for supporting thework of the user. The work tool 64A displays a moving image, ananimation, or the like that conveys the work procedure. The work tool64B is provided with commands for operation. The work description 63 andthe work tool 64 are generated by, for example, a work supportapplication. The work tool 64 is an example of display with reference tothe neck coordinate system CN, and is displayed at a position alignedwith the body trunk direction of the user. When the positionalrelationship between the neck attached part 1B and the head attachedpart 1A changes, the work tool 64 changes its display position withinthe entire visual field 5A. Hereinafter, a change of the display of thework tool 64 with reference to the neck coordinate system CN will bedescribed.

FIG. 16 shows a second state which is an example of a state changed fromthe first state of FIG. 15. FIG. 16 shows a state in which a front ofthe user's face is directed toward the work object 5D by the userturning the head diagonally downward while the direction of the user'sbody trunk is kept as it is. The work description 63 is present at thepoint P1 of the FOV 5B. The work tool 64 has moved upward in terms ofthe visual field and is completely out of the FOV 5B.

FIG. 17 shows a third state which is an example of a state changed fromthe first state of FIG. 15. FIG. 17 is a state in which the work tool 64is directed to a lower-right direction in the entire visual field 5B anda left work tool 64A is accommodated in the FOV 5B by the user turningthe face toward an upper-left direction while the direction of the user′body trunk is kept as it is. The work tool 64A is present at the pointP1.

FIG. 18 shows a fourth state which is an example of a state changed fromthe first state of FIG. 15. FIG. 18 is a state in which the user turnsthe body trunk diagonally forward left without changing the direction ofthe face with respect to the body trunk. In this case, the workdescription 63, which is a display object with reference to the worldcoordinate system CW, is moving to the right within the entire visualfield 5A. Meanwhile, since the work tool 64, which is a display objectwith reference to the neck coordinate system CN, is arranged so as to bealigned with the body trunk direction, the position is not changedwithin the entire visual field 5A. Such display with reference to theneck coordinate system CN is suitable for, for example, a case etc.where the user uses the same work tool 64 while moving.

Like the above-mentioned example, as a method of the display control ofthe virtual object in the HMD 1 of the fifth embodiment, the display ofthe virtual objects with reference to the respective coordinate systemsof the world coordinate system CW, head coordinate system CH, and neckcoordinate system CN can be displayed, selected, and used in combinationbased on data of the detected and calculated states. In particular, adisplay method with reference to the neck coordinate system CN can bedisplayed so as to follow the movements of the user and the HMD 1, butnot to follow the changes in the directions of the head attached part 1Aand the head with respect to the arrangement position of the virtualobject. This makes it possible to perform detailed display according tothe type and nature of the display object and to improve conveniencesuch as preferable support of the user's work.

[Display Control Example (2)]

FIGS. 19 to 20 show a second display control example of the fifthembodiment. FIG. 19 shows an example in which a plurality of applicationicons 66 are displayed in the FOV 5B on the display surface 5 of the HMD1 by using a method with reference to the head coordinate system CH. Inthis example, the HMD 1 opens the home screen and displays a pluralityof (8 in this example) application icons 66 in parallel on the homescreen in response to the user selecting and executing the HOME buttonin the HMD menu 62 described above. The application icon 66 is anexample of a virtual object that is a GUI component, and is an icon forcontrolling the display or the like of the corresponding applicationaccording to the selection execution by the user. Normally, the HMD 1uses a method of displaying the application icon 66 at a fixed positionin the FOV 5B with reference to the head coordinate system CH. When theuser selects and executes a CLOSE button in the HMD menu 62, the HMDreturns to an original display state, that is, a state where theapplication icon 66 is not displayed. For the virtual objects such asthe application icon 66 and the like that are frequently used by theuser, it is more convenient to display them at the fixed positions inthe FOV 5B in this way.

Incidentally, display of a display object that interferes with thedisplay of the application icon 66 may be temporarily stopped, or theapplication icon 66 may be superimposedly displayed on an upper layer.Further, for example, used may be a method in which an icon is placed oneach page of a plurality of pages in the FOV 5B and a page changeoperation is accepted by a one-page change button or the like of the HMDmenu 62. This makes it possible to increase the number of iconscomprehensively displayable on the display surface 5.

Further, FIG. 20 shows an example of displaying the application icon 66with reference to the neck coordinate system CN. The HMD 1 arranges aplurality of application icons 66 in parallel in the entire visual field5A so as to aligned with the body trunk direction. Some of theapplication icons 66 are displayed in the FOV 5B. The application icon66 displayed in the FOV 5B can be changed by the user changing theface's direction while the body trunk direction is kept as it is. Forexample, when the user turns the face to the upper left, the applicationicon 66 of “A” enters the FOV 5B. Thus, in the display method withreference to the neck coordinate system CN, all of a larger number ofapplication icons 66 than a conventional application icon(s) can bevisually recognized in the FOV 5B by changing the head direction withrespect to the body trunk direction.

[Display Control Example (3)]

FIG. 21 shows a third display control example of the fifth embodiment.Further, used may be an application-icon arrangement method withreference to the world coordinate system CW. For example, since anapplication for operating a specific apparatus or device is often usedat a location of the application, it is more convenient to place avirtual object of the application at a position near the application inthe world coordinate system CW.

(A) of FIG. 21 shows an example in which an application icon 67 forcontrolling an application for operating a work object 5D is displayedat a fixed position near the work object 5D on the work table 5C in theworld coordinate system CW within the entire visual field 5A.

Further, at that time, arrangement of the application icon 67 as adisplay object in the world coordinate system CW may be arrangement at aposition and in a direction with reference to a direction in which theuser exists. Furthermore, at that time, the application icon 67 may bearranged at a position avoiding being arranged on a line that connectsthe user and a work place of the work object 5D so as not to interferewith the user's work. In (B) of FIG. 21, an arrangement example of theapplication icon 67 is shown as a positional relationship seen fromvertically upward. A direction 2101 of an arrow of a dash-single-dotline is a direction that connects a representative point of the user andHMD 1 and a representative point of the work object 5D and thatcorresponds to the head direction or the body trunk direction. Adirection 2102 of a broken-line arrow is a direction that connects therepresentative point of the user and HMD 1 and an arrangement positionof the application icon 67. The arrangement position of the applicationicon 67 is set, for example, at a right-hand neighbor position so as notto overlap with the work object 5D. An arrangement direction of theapplication icon 67 is aligned with the direction 2102.

[Display Control Example (4)]

FIG. 22 shows a fourth display control example of the fifth embodiment.A display method with reference to each of the above-mentionedcoordinate systems may be selectable according to a user's instructionor preset setting, or may be selected by HMD1's judgment. A displaymethod may be selectable for each target virtual object, each type,and/or each application. FIG. 22 shows a setting example of a coordinatesystem used for displaying an application. FIG. 22 displays of aproperty (corresponding virtual object) of a certain application, andshows an example in which a coordinate system for displaying a virtualobject such as an icon or tool of the application is selected from thehead coordinate system CH, the neck coordinate system CN, and the worldcoordinate system CW and is settable.

Further, FIG. 23 shows an example in which an image representing adisplay coordinate system is superimposedly displayed on a virtualobject. In FIG. 23, an image 68 representing a display coordinate systemis superimposedly displayed on a part of an application icon 66. Theimage 68 of this example is an image representing the world coordinatesystem CW, and indicates that the display coordinate system of theapplication icon 66 is the world coordinate system CW. The image 68 mayalways be displayed, or the image 68 may be displayed in a case or thelike where the user provisionally selects the application icon 66 orinputs a predetermined instruction.

Furthermore, this image 68 representing the display coordinate systemmay be used as a control button of the display coordinate system. TheHMD 1 is set to switch the display coordinate system of thecorresponding virtual object among the head coordinate system CH, theneck coordinate system CN, and the world coordinate system CW accordingto selection execution of the image 68 by the user. For example, theimage 68 may be a button that is cyclically switched among the threecoordinate systems. For example, a selected state of the image 68 of theapplication icon 66 represents a display coordinate system at a time ofstarting the corresponding application. When the application icon 66 isselected and executed, the HMD 1 starts the corresponding application ata position of the display coordinate system according to the selectedstate of the image 68 at that time. That is, a virtual object such as awork tool is arranged at a position of the display coordinate system. Inaddition, a selecting operation of the image 68 of the application icon66 may be used for immediately switching of the display coordinatesystem. For example, first, a state of the image 68 of the applicationicon 66 is in the head coordinate system CH, and is displayed like theexample of FIG. 19 described above. Next, the state of the image 68 ischanged to the neck coordinate system CN by the user's selectingoperation of the image 68. At this time, the HMD 1 changes thecoordinate system and the position, which the application icon 66 isarranged in and at, to a position in the neck coordinate system CN. Thechange of the coordinate system is possible based on the relationshipbetween the coordinate systems described above.

(A) of FIG. 24 shows an example of switching a display coordinate systemof a virtual object in the above-mentioned example of the work tool 64A.This example is an example of switching the display coordinate systemaccording to user's selection execution for a virtual object such as abutton arranged in the vicinity of the virtual object. In the work tool64A, a control display object 64 b that enables an operation such asswitching of the display coordinate system is displayed at a positionadjacent to a region of the display object 64 a such as a moving imagethat conveys the work procedure, for example, in an upper-side area. Thedisplay object 64 b includes a button 64 c that enables the operation ofswitching the display coordinate system, a reduction button 64 dindicated by “-”, and an end button 64 e indicated by “x”.

The button 64 c is a display coordinate system switching button, and maybe a cyclic switching button similarly to the image 68 of FIG. 23. Thebutton 64 c includes an image (corresponding button) representing eachcoordinate system of the head coordinate system CH, the neck coordinatesystem CN, and the world coordinate system CW. The image correspondingto the currently selected display coordinate system is highlighted anddisplayed by, for example, being surrounded with a frame. The user canswitch the display coordinate system of the work tool 64A by selectingand operating a button of the desired display coordinate system.

As an example of using the display coordinate system switching, the userchanges the display coordinate system of a plurality of display objectsarranged at a certain place in the world coordinate system CW to theneck coordinate system CN or the head coordinate system CH. The user cancarry those display objects by moving from that place. Then, the userreturns, at a moved place, the display coordinate system of thosedisplay objects to, for example, the world coordinate system CW. Thismakes it possible for the user to change the position of each displayobject in the world coordinate system CW with little time and effort.

Reduced display during transportation or for a display object(s) thatdoes not need to be watched for a while brings efficient availability ofa displace space. The HMD 1 changes a display size of the virtual objectaccording to a user's instruction or the like. (B) of FIG. 24 shows anexample in which the work tool 64A is reduced and displayed by operatingthe reduction button 64 d of (A). One of the control buttons is anenlargement button 64 f illustrated by a rectangle, and the work tool64A can be returned to an original state by operating the enlargementbutton 64 f. The present embodiment can adopt an aspect of not only thereduced display but also a change or the like of the virtual object bythe icon. As described above, the function of switching the displaycoordinate system for each virtual object can enhance the convenience ofthe user.

[Display Control Example (5)]

FIG. 25 shows a fifth display control example of the fifth embodiment.When the positional relationship between the head attached part 1A andthe neck attached part 1B is unknown, a coordinate system used as analternative to the neck coordinate system CN may be provided. Here, thiscoordinate system is called an inertial coordinate system CI.Specifically, this inertial coordinate system CI is a coordinate systemconfigured so that the coordinate origin of the world coordinate systemCW follows the movement of the head coordinate system CH (correspondinghead attached part 1A) or the neck coordinate system CN (correspondingneck attached part 1B). For example, at a certain point, a displayobject placed at a position on the inertial coordinate system CI in thenorth direction seen from the user is always displaced at a positiondirected in the north direction with respect to the user regardless ofthe user's movement or a change of the user's direction.

Such a display method with reference to the inertial coordinate systemCI is also a display method in which an arrangement position of thevirtual object follows the movement of the user and the HMD 1 but doesnot follow the change in the direction of the head attached part 1A. Ifthe front direction of the user's body (the body trunk direction of theneck coordinate system CN) is unknown, use of the inertial coordinatesystem CI has an effect in which the virtual objects desired to beplaced in the vicinity of the user can be placed in a wide region aroundthe user. This display method is applicable to any HMD that can measurethe world coordinate system CW.

The HMD 1 switches the display coordinate system of the display object,which is arranged in the neck coordinate system CN, to the inertialcoordinate system CI, for example, when the positional relationshipbetween the head attached part 1A and the neck attached part 1B istemporarily unknown due to some causes and the direction of the neckcoordinate system CN is unknown. At the time when the direction of theneck coordinate system CN becomes known again, the HMD 1 returns thedisplay coordinate system of the display object from the inertialcoordinate system CI to the neck coordinate system CN. Alternatively,the HMD 1 may select and switch the inertial coordinate system CI as thedisplay coordinate system of the target virtual object or the like inresponse to the user's instruction (for example, the same operation asthat of the fourth display control example).

FIG. 25 shows a concept of the inertial coordinate system CI. First, theuser who wears (attaches) the HMD 1 is present at a position P01 in theworld coordinate system CW. The directions of the user's body trunk andhead are, for example, the north direction. Virtual objects 251,252,253, etc. can be seen in the FOV 25A corresponding to the northdirection. Three axes of the inertial coordinate system CI are shown by(X_(I), Y_(I), Z_(I),). The axis Z_(I) corresponds to a verticaldirection, the axis X_(I) corresponds to the north direction, and theaxis Y_(I) corresponds to the west direction. The HMD 1 changes thedisplay coordinate system of the virtual object 251 or the like from,for example, the neck coordinate system CN to the inertial coordinatesystem CI at a predetermined opportunity.

A case where the user moves from the position P01 and changes the headdirection so as to rotate is shown at a position P02. The head directionis rotated about 45 degrees from the north direction to the right side,for example. The origin of the inertial coordinate system CI has movedfrom the position P01 to the position P02. The virtual object 251 andthe like are moving together with the inertial coordinate system CI, andare arranged at positions in the north direction while the positionalrelationship from the user's position P02 is maintained. In the FOV 25Bin the head direction at the position P02, the virtual object 251 is notvisible and the virtual object 252 is visible. The HMD 1 returns thedisplay coordinate system of the virtual object 251 or the like from theinertial coordinate system CI to, for example, the neck coordinatesystem CN at a predetermined opportunity. As described above,introducing a new coordinate system makes it possible to control thedisplay of the virtual objects under various conditions, and to furtherimprove the convenience of the user.

Although the present invention has been specifically described abovebased on the embodiments, the present invention is not limited to theabove-described embodiments and can be variously modified withoutdeparting from the scope.

EXPLANATION OF REFERENCE NUMERALS

1 . . . HMD; 1A . . . Head attached part; 1B . . . Neck attached part; 2. . . Operating tool (remote control); 3A . . . Server; 3B . . . PC; 4 .. . Connecting line; 5 . . . Display surface; 6 . . . Camera; 7 . . .Distance measuring sensor; 8 . . . Microphone; 9 . . . Speaker; 10, 11,12 . . . Housing; 14 . . . Operation input unit; and 17 . . . Marker.

1. A head mounted display apparatus capable of displaying an image in auser's field of view, the apparatus comprising: a head attached partattached to a user's head and having a display surface for displayingthe image; and a body truck attached part communicating with the headattached part and attached to a part of a user's body truck, wherein theapparatus measures a relatively positional relationship between the headattached part and the body trunk attached part and, based on thepositional relationship, grasps states including a position and adirection of the head attached part and states including a position anda direction of the body trunk attached part.
 2. The head mounted displayapparatus according to claim 1, wherein the body trunk attached part isattached near a user's neck or shoulder, and has housings arranged onright and left sides with respect to the user's neck or shoulder, and ahousing arranged on a back side with respect to the user's neck orshoulder so as to connect the right-side and left-side housings.
 3. Thehead mounted display apparatus according to claim 1, wherein the bodytrunk attached part: has at least apart of a sensor for detecting statesincluding a position and a direction of the head mounted displayapparatus, and calculates the states including the position and thedirection of the head mounted display apparatus based on informationdetected by the sensor and the positional relationship, and controlsdisplay of the image according to the states.
 4. The head mounteddisplay apparatus according to claim 1, wherein at least one of the headattached part and the body trunk attached part has a distance measuringsensor for measuring the positional relationship.
 5. The head mounteddisplay apparatus according to claim 4, wherein positions of a pluralityof markers provided to a housing of the head attached part are measuredfrom the distance measuring sensor that the body trunk attached parthas, or positions of a plurality of markers provided to the housings ofthe body trunk attached part are measured from the distance measuringsensor that the head attached part has.
 6. The head mounted displayapparatus according to claim 1, wherein the body trunk attached part hasa body trunk sensor for detecting states including a position and adirection of the body trunk attached part, and the body trunk attachedpart measures the positional relationship, calculates the statesincluding the position and direction of the head attached part based oninformation detected by the body trunk sensor and the positionalrelationship, and performs display control of the image, which uses thestates including the position and the direction of the body trunkattached part, and display control of the image, which uses the statesincluding the position and the direction of the head attached part. 7.The head mounted display apparatus according to claim 1, wherein thehead attached part has ahead sensor for detecting the states includingthe position and the direction of the head attached part, and the headattached part measures the positional relationship, calculates thestates including the position and the direction of the body trunkattached part based on information detected by the head sensor and thepositional relationship, and performs display control of the image,which uses the states including the position and the direction of thehead attached part, and display control of the image, which uses thestates including the position and the direction of the body trunkattached part.
 8. The head mounted display apparatus according to claim1, wherein the body trunk attached part has a main controller, and themain controller grasps the states based on the positional relationshipand transmits, to the head attached part, display data for the displaycontrol of the image according to the states.
 9. The head mounteddisplay apparatus according to claim 1, further comprising a connectingline that connects the body trunk attached part and the head attachedpart, wherein the connecting line has two connecting lines arrangedsymmetrically with respect to the body trunk of the user.
 10. The headmounted display apparatus according to claim 1, wherein the body trunkattached part has main batteries at the right-side and left-sidehousings with respect to the body trunk of the user, and includes aconnecting line that connects the body trunk attached part and the headattached part, power being fed from the body trunk attached part throughthe connecting line.
 11. The head mounted display apparatus according toclaim 1, wherein the body trunk attached part has microphones andspeakers at right-side and left-side housings with respect to the bodytrunk of the user.
 12. The head mounted display apparatus according toclaim 1, further comprising a connecting line that connects the bodytrunk attached part and the head attached part, wherein wiredcommunication is performed between a wired communication circuit of thebody trunk attached part and a wired communication circuit of the headattached part through the connecting line.
 13. The head mounted displayapparatus according to claim 1, wherein wireless communication isperformed between a wireless communication circuit of the body trunkattached part and a wireless communication circuit of the head attachedpart.
 14. The head mount display apparatus according to claim 1, furthercomprising a connecting line that connects the body trunk attached partand the head attached part, wherein the head attached part has a powerreception circuit for the wireless power feeding, the connecting linehas a power reception antenna unit for the wireless power feeding, andthe power transmission antenna unit and the power reception antenna arearranged close to each other, and the wireless power feeding isperformed between the power transmission antenna unit and the powerreception antenna unit.
 15. The head mounted display apparatus accordingto claim 1, wherein the body trunk attached part or the head attachedpart has a camera that captures a field of view of the user.
 16. Thehead mounted display apparatus according to claim 1, further comprising:as reference coordinate systems in arranging and displaying the image onthe display surface, a world coordinate system, a body trunk coordinatesystem fixed to the body trunk attached part, and a head coordinatesystem fixed to the head attached part, wherein the reference coordinatesystems are selectable and settable for each of the images, each type ofthe images, or each of applications that handles the images.
 17. Thehead mounted display apparatus according to claim 16, further comprisingan inertial coordinate system as the reference coordinate system,wherein the inertial coordinate system is a coordinate system in whichits origin moves so as to follow a change in the position of the headattached part or the body trunk attached part and does not follow anarrangement position of the image with respect to a change in thedirections of the head attached part and the body trunk attached part.